Skip to content

proofs: expose native EVMYulLean transition target#1743

Open
Th0rgal wants to merge 186 commits intomainfrom
codex/native-evmyullean-transition
Open

proofs: expose native EVMYulLean transition target#1743
Th0rgal wants to merge 186 commits intomainfrom
codex/native-evmyullean-transition

Conversation

@Th0rgal
Copy link
Copy Markdown
Member

@Th0rgal Th0rgal commented Apr 22, 2026

Summary

  • expose Native.interpretIRRuntimeNative as the executable native EVMYulLean target for emitted IR runtime Yul
  • preserve pre-existing event history when projecting native EVMYulLean results back to YulResult
  • close the executable/source-surface migration blockers reported against the transition: environment accessors, mapping-struct storage agreement, overloaded function identity, direct helper-call lowering, packed full-word storage writes, and dynamic arrays of static tuple elements
  • document the clean transition plan from the backend-parameterized Verity interpreter to native EvmYul.Yul.callDispatcher

Fixed issues

Closes #1738.
Closes #1739.
Closes #1740.
Closes #1741.
Closes #1742.
Closes #1744.
Closes #1745.

Refs #1737.
Refs #1722.

Why

PR #1735 was only a docs/sync cleanup and did not close the native-runtime work from #1722. The current public theorem target still runs through interpretYulRuntimeWithBackend .evmYulLean, which uses Verity's custom Yul statement interpreter with EVMYulLean-backed builtins. That bridge is useful, but it is not the simpler final architecture requested in #1737.

This PR is now the native transition foundation plus the reported frontend/executable parity fixes needed before the remaining work can focus on the native public theorem target.

What changed

  • Added the native IR-runtime harness entry point Native.interpretIRRuntimeNative over EvmYul.Yul.callDispatcher.
  • Added result/state projection coverage for storage, events, environment fields, halt/revert/error behavior, and explicit observable storage slots.
  • Fixed verity_contract environment reads such as blockTimestamp, blockNumber, chainid, blobbasefee, contractAddress, msgSender, and msgValue, including executable .run state reads.
  • Aligned mapping-struct executable helpers with the compiler/native abstract mapping-slot formula, including packed member masking.
  • Introduced signature-based function identity for overloads while preserving Solidity-facing names for selectors and ABI dispatch.
  • Confirmed direct function-name calls are the user-facing internal-helper surface; explicit internalCall/internalCallAssign remain only lower-level compilation-model IR.
  • Added first-class packed full-word storage writes through setPackedStorage root offset word, lowering to Stmt.setStorageWord / sstore(root.slot + offset, word).
  • Added dynamic array support for static tuple elements on arrayLength/arrayElement; dynamic element arrays such as Array String and Array Bytes still fail closed.
  • Updated docs and CI guards so the repo does not claim native EVMYulLean is authoritative before the public theorem target flips.

Remaining work for #1737

  • prove native lowering invariants for dispatcher/function partitioning and builtin primop lowering
  • prove native state bridge lemmas for calldata, selector, caller/source, callvalue, address, block fields, storage, transient storage, memory, and returndata
  • prove result projection lemmas for normal values, stop, return, revert rollback, logs, and hard native errors
  • widen executable native coverage for dispatcher selection, memory-heavy return/revert/log paths, returndata, external calls, static-call permissions, and mapping helper lowering
  • introduce a public preservation theorem targeting Native.interpretIRRuntimeNative or a total wrapper around it
  • only then move execYulFuel / execYulFuelWithBackend to reference-oracle status and update trust-boundary docs accordingly

Validation

  • lake build Contracts.Smoke
  • lake build Contracts.MacroTranslateInvariantTest
  • lake build Contracts.MacroTranslateRoundTripFuzz
  • lake build Compiler.Proofs.YulGeneration.Backends.EvmYulLeanNativeHarness Compiler.Proofs.YulGeneration.Backends.EvmYulLeanNativeSmokeTest
  • python3 scripts/check_selectors.py
  • python3 -m unittest scripts/test_native_transition_api.py scripts/test_check_native_transition_doc.py scripts/test_check_selectors.py -v
  • python3 -m unittest scripts/test_generate_macro_property_tests.py -v
  • python3 scripts/check_macro_property_test_generation.py
  • python3 scripts/check_lean_hygiene.py
  • python3 scripts/check_verification_status_doc.py
  • python3 scripts/check_layer2_boundary_sync.py
  • python3 scripts/check_layer2_boundary_catalog_sync.py
  • python3 scripts/check_docs_workflow_sync.py
  • git diff --check
  • make check

Note

Medium Risk
Adds new compiler IR constructs and modifies dispatch/ABI helper generation plus large proof-surface changes around a new native EVMYulLean execution seam; mistakes could change emitted Yul/storage semantics or invalidate conformance guarantees.

Overview
Exposes a native EVMYulLean execution target for emitted runtime Yul by extending Compiler/Proofs/EndToEnd.lean with new comparison surfaces (nativeResultsMatchOn, yulResultsAgreeOn) and explicit bridge obligations that relate Native.interpretIRRuntimeNative/callDispatcher results to the existing EVMYulLean-backed interpreter oracle.

Extends the compilation model and codegen to support (1) word-level access into dynamic-array elements via Expr.arrayElementWord plus new checked helper functions, and (2) full-word storage writes at field.slot + offset via Stmt.setStorageWord (including mirroring alias slots). It also refines ABI/dynamic-array bounds checks to account for multi-word static elements.

Tightens validation/dispatch rules by treating external overloads as unique by function signature (not just name), adds explicit checks for duplicate/colliding internal function names, and updates reserved-symbol collision detection to include the new helper names. Updates CI/docs conformance guard messaging to include the native harness/oracle coverage and adds targeted smoke tests/regressions for the new features and name-collision diagnostics.

Reviewed by Cursor Bugbot for commit 019a943. Bugbot is set up for automated code reviews on this repo. Configure here.

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: fc6b2b9c8a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread Compiler/Proofs/YulGeneration/Backends/EvmYulLeanNativeHarness.lean Outdated
Comment thread Compiler/Proofs/IRGeneration/FunctionBody.lean Outdated
@Th0rgal Th0rgal force-pushed the codex/native-evmyullean-transition branch from 0672929 to 39ae307 Compare April 24, 2026 16:47
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 39ae307375

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread Compiler/CompilationModel/Compile.lean Outdated
Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 4f58dad. Configure here.

Comment thread Compiler/CompilationModel/Compile.lean Outdated
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b28b535b6a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread Compiler/Proofs/IRGeneration/SourceSemantics.lean Outdated
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c62fc97e30

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +90 to +94
def materializedStorageSlots
(runtimeCode : List YulStmt)
(observableSlots : List Nat) :
List Nat :=
yulStmtsLiteralStorageReadSlots runtimeCode ++ observableSlots
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Materialize computed storage read slots for native initial state

materializedStorageSlots only preloads literal sload slots plus caller-supplied observableSlots, but the emitted runtime frequently reads storage through computed slots (for example mapping/dynamic-array paths lowered to sload(mappingSlot(...))). Those computed read slots are absent from initialState, so native execution sees them as zero and can take different branches or write different results than source semantics unless the caller manually precomputes every read slot. This makes interpretRuntimeNative behavior input-dependent on an incomplete preload set and can silently corrupt native/source equivalence for contracts that branch on mapping values.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b462beb0c2

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Used by `modifies(...)` validation (#1729, Axis 3 Step 1b). -/
def stmtWrittenFields : Stmt → List String
| Stmt.setStorage field _ | Stmt.setStorageAddr field _
| Stmt.setStorage field _ | Stmt.setStorageAddr field _ | Stmt.setStorageWord field _ _
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Validate setStorageWord against modifies using resolved slots

stmtWrittenFields treats Stmt.setStorageWord as a write to just the root field name, but lowering executes sstore(field.slot + wordOffset, ...) (and alias mirrors), so nonzero offsets can mutate different declared fields. A function annotated modifies(root) can therefore pass validation while writing outside the declared set (for example, setPackedStorage root 1 ... clobbering the next slot), which makes the generated modifies/frame guarantees unsound. The modifies check should either reject nonzero wordOffset here or compute touched fields from resolved slots instead of the root field label.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 81b7d3478b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +197 to +199
match fn.params.toList.map (fun p => valueTypeSignatureComponent p.ty) with
| [] => "0"
| parts => String.intercalate "_" parts
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Make overload suffix encoding unambiguous

overloadedFunctionIdentName builds suffixes by joining per-parameter type components with _, but those components already contain underscores, so the mapping is not injective. A legal overload set can therefore collapse to the same generated identifier (e.g., one-arg newtype_n_scalar_uint256 over Uint8 vs. two args newtype_n over Uint256 and Uint8 both produce newtype_n_scalar_uint256_scalar_uint8), and validateGeneratedDefNamesPublic then rejects it as a duplicate internal declaration. This turns valid overloads into deterministic compile failures; the suffix encoding needs explicit boundaries (such as length-prefixing each component).

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment